Reconstructing Dynamical Systems From Stochastic Differential Equations to Machine Learning

Die Modellierung komplexer Systeme mit einer großen Anzahl von Freiheitsgraden ist in den letzten Jahrzehnten zu einer großen Herausforderung geworden. In der Regel werden nur einige wenige Variablen komplexer Systeme in Form von gemessenen Zeitreihen beobachtet, während die meisten von ihnen - die möglicherweise mit den beobachteten Variablen interagieren - verborgen bleiben. In dieser Arbeit befassen wir uns mit dem Problem der Rekonstruktion und Vorhersage der zugrunde liegenden Dynamik komplexer Systeme mit Hilfe verschiedener datengestützter Ansätze. Im ersten Teil befassen wir uns mit dem umgekehrten Problem der Ableitung einer unbekannten Netzwerkstruktur komplexer Systeme, die Ausbreitungsphänomene widerspiegelt, aus beobachteten Ereignisreihen. Wir untersuchen die paarweise statistische Ähnlichkeit zwischen den Sequenzen von Ereigniszeitpunkten an allen Knotenpunkten durch Ereignissynchronisation (ES) und Ereignis-Koinzidenz-Analyse (ECA), wobei wir uns auf die Idee stützen, dass funktionale Konnektivität als Stellvertreter für strukturelle Konnektivität dienen kann. Im zweiten Teil konzentrieren wir uns auf die Rekonstruktion der zugrunde liegenden Dynamik komplexer Systeme anhand ihrer dominanten makroskopischen Variablen unter Verwendung verschiedener stochastischer Differentialgleichungen (SDEs). In dieser Arbeit untersuchen wir die Leistung von drei verschiedenen SDEs - der Langevin-Gleichung (LE), der verallgemeinerten Langevin-Gleichung (GLE) und dem Ansatz der empirischen Modellreduktion (EMR). Unsere Ergebnisse zeigen, dass die LE bessere Ergebnisse für Systeme mit schwachem Gedächtnis zeigt, während sie die zugrunde liegende Dynamik von Systemen mit Gedächtniseffekten und farbigem Rauschen nicht rekonstruieren kann. In diesen Situationen sind GLE und EMR besser geeignet, da die Wechselwirkungen zwischen beobachteten und unbeobachteten Variablen in Form von Speichereffekten berücksichtigt werden. Im letzten Teil dieser Arbeit entwickeln wir ein Modell, das auf dem Echo State Network (ESN) basiert und mit der PNF-Methode (Past Noise Forecasting) kombiniert wird, um komplexe Systeme in der realen Welt vorherzusagen. Unsere Ergebnisse zeigen, dass das vorgeschlagene Modell die entscheidenden Merkmale der zugrunde liegenden Dynamik der Klimavariabilität erfasst.Modeling complex systems with large numbers of degrees of freedom have become a grand challenge over the past decades. Typically, only a few variables of complex systems are observed in terms of measured time series, while the majority of them – which potentially interact with the observed ones - remain hidden. Throughout this thesis, we tackle the problem of reconstructing and predicting the underlying dynamics of complex systems using different data-driven approaches. In the first part, we address the inverse problem of inferring an unknown network structure of complex systems, reflecting spreading phenomena, from observed event series. We study the pairwise statistical similarity between the sequences of event timings at all nodes through event synchronization (ES) and event coincidence analysis (ECA), relying on the idea that functional connectivity can serve as a proxy for structural connectivity. In the second part, we focus on reconstructing the underlying dynamics of complex systems from their dominant macroscopic variables using different Stochastic Differential Equations (SDEs). We investigate the performance of three different SDEs – the Langevin Equation (LE), Generalized Langevin Equation (GLE), and the Empirical Model Reduction (EMR) approach in this thesis. Our results reveal that LE demonstrates better results for systems with weak memory while it fails to reconstruct underlying dynamics of systems with memory effects and colored-noise forcing. In these situations, the GLE and EMR are more suitable candidates since the interactions between observed and unobserved variables are considered in terms of memory effects. In the last part of this thesis, we develop a model based on the Echo State Network (ESN), combined with the past noise forecasting (PNF) method, to predict real-world complex systems. Our results show that the proposed model captures the crucial features of the underlying dynamics of climate variability

Reconstructing complex system dynamics from time series: a method comparison

Modeling complex systems with large numbers of degrees of freedom has become a grand challenge over the past decades. In many situations, only a few variables are actually observed in terms of measured time series, while the majority of variables - which potentially interact with the observed ones - remain hidden. A typical approach is then to focus on the comparably few observed, macroscopic variables, assuming that they determine the key dynamics of the system, while the remaining ones are represented by noise. This naturally leads to an approximate, inverse modeling of such systems in terms of stochastic differential equations (SDEs), with great potential for applications from biology to finance and Earth system dynamics. A well-known approach to retrieve such SDEs from small sets of observed time series is to reconstruct the drift and diffusion terms of a Langevin equation from the data-derived Kramers-Moyal (KM) coefficients. For systems where interactions between the observed and the unobserved variables are crucial, the Mori-Zwanzig formalism (MZ) allows to derive generalized Langevin equations that contain non-Markovian terms representing these interactions. In a similar spirit, the empirical model reduction (EMR) approach has more recently been introduced. In this work we attempt to reconstruct the dynamical equations of motion of both synthetical and real-world processes, by comparing these three approaches in terms of their capability to reconstruct the dynamics and statistics of the underlying systems. Through rigorous investigation of several synthetical and real-world systems, we confirm that the performance of the three methods strongly depends on the intrinsic dynamics of the system at hand. For instance, statistical properties of systems exhibiting weak history-dependence but strong state-dependence of the noise forcing, can be approximated better by the KM method than by the MZ and EMR approaches. In such situations, the KM method is of a considerable advantage since it can directly approximate the state-dependent noise. However, limitations of the KM approximation arise in cases where non-Markovian effects are crucial in the dynamics of the system. In these situations, our numerical results indicate that methods that take into account interactions between observed and unobserved variables in terms of non-Markovian closure terms (i.e., the MZ and EMR approaches), perform comparatively better. © 2020 The Author(s). Published by IOP Publishing Ltd on behalf of the Institute of Physics and Deutsche Physikalische Gesellschaf

Long-term ENSO prediction with echo-state networks

The El Niño-Southern Oscillation (ENSO) is a climate phenomenon that profoundly impacts weather patterns and extreme events worldwide. Here we develop a method based on a recurrent neural network, called echo state network (ESN), which can be trained efficiently to predict different ENSO indices despite their relatively high noise levels. To achieve this, we train the ESN model on the low-frequency variability of ENSO indices and estimate the potential future high-frequency variability from specific samples of its past history. Our method reveals the importance of cross-scale interactions in the mechanisms underlying ENSO and skilfully predicts its variability and especially El Niño events at lead times up to 21 months. This study considers forecasts skillful if the correlation coefficients are above 0.5. Our results show that the low-frequency component of ENSO carries substantial predictive power, which can be exploited by training our model on single scalar time series. The proposed machine learning method for data-driven modeling can be readily applied to other time series, e.g. finance and physiology. However, it should be noted that our approach cannot straightforwardly be turned into a real-time operational forecast because of the decomposition of the original time series into the slow and fast components using low-pass filter techniques.Volkswagen foundation and the European Union’s Horizon 2020 research and innovation programmeGerman Academic Exchange Service (DAAD)Marie Sklodowska-Curie grant agreementFederal Ministry of Education and Research under grantFederal Ministry of Education and ResearchPeer Reviewe

Stable stadial and interstadial states of the last glacial's climate identified in a combined stable water isotope and dust record from Greenland

During the last glacial interval, the Northern Hemisphere climate was punctuated by a series of abrupt changes between two characteristic climate regimes. The existence of stadial (cold) and interstadial (milder) periods is typically attributed to a hypothesised bistability in the glacial North Atlantic climate system, allowing for rapid transitions from the stadial to the interstadial state – the so-called Dansgaard–Oeschger (DO) events – and more gradual yet still fairly abrupt reverse shifts. The physical mechanisms driving these regime transitions remain debated. DO events are characterised by substantial warming over Greenland and a reorganisation of the Northern Hemisphere atmospheric circulation, which are evident from concomitant shifts in the δ18O ratios and dust concentration records from Greenland ice cores. Treating the combined δ18O and dust record obtained by the North Greenland Ice Core Project (NGRIP) as a realisation of a two-dimensional, time-homogeneous, and Markovian stochastic process, we present a reconstruction of its underlying deterministic drift based on the leading-order terms of the Kramers–Moyal equation. The analysis reveals two basins of attraction in the two-dimensional state space that can be identified with the stadial and interstadial regimes. The drift term of the dust exhibits a double-fold bifurcation structure, while – in contrast to prevailing assumptions – the δ18O component of the drift is clearly mono-stable. This suggests that the last glacial's Greenland temperatures should not be regarded as an intrinsically bistable climate variable. Instead, the two-regime nature of the δ18O record is apparently inherited from a coupling to another bistable climate process. In contrast, the bistability evidenced in the dust drift points to the presence of two stable circulation regimes of the last glacial's Northern Hemisphere atmosphere.Stable stadial and interstadial states of the last glacial's climate identified in a combined stable water isotope and dust record from GreenlandpublishedVersio

Reconstructing complex system dynamics from time series: a method comparison

Modeling complex systems with large numbers of degrees of freedom has become a grand challenge over the past decades. In many situations, only a few variables are actually observed in terms of measured time series, while the majority of variables—which potentially interact with the observed ones—remain hidden. A typical approach is then to focus on the comparably few observed, macroscopic variables, assuming that they determine the key dynamics of the system, while the remaining ones are represented by noise. This naturally leads to an approximate, inverse modeling of such systems in terms of stochastic differential equations (SDEs), with great potential for applications from biology to finance and Earth system dynamics. A well-known approach to retrieve such SDEs from small sets of observed time series is to reconstruct the drift and diffusion terms of a Langevin equation from the data-derived Kramers–Moyal (KM) coefficients. For systems where interactions between the observed and the unobserved variables are crucial, the Mori–Zwanzig formalism (MZ) allows to derive generalized Langevin equations that contain non-Markovian terms representing these interactions. In a similar spirit, the empirical model reduction (EMR) approach has more recently been introduced. In this work we attempt to reconstruct the dynamical equations of motion of both synthetical and real-world processes, by comparing these three approaches in terms of their capability to reconstruct the dynamics and statistics of the underlying systems. Through rigorous investigation of several synthetical and real-world systems, we confirm that the performance of the three methods strongly depends on the intrinsic dynamics of the system at hand. For instance, statistical properties of systems exhibiting weak history-dependence but strong state-dependence of the noise forcing, can be approximated better by the KM method than by the MZ and EMR approaches. In such situations, the KM method is of a considerable advantage since it can directly approximate the state-dependent noise. However, limitations of the KM approximation arise in cases where non-Markovian effects are crucial in the dynamics of the system. In these situations, our numerical results indicate that methods that take into account interactions between observed and unobserved variables in terms of non-Markovian closure terms (i.e., the MZ and EMR approaches), perform comparatively better.Peer Reviewe